Skip to main content

Show sorted alphabetically
More right-solid
Show sorted alphabetically
SHOW DETAILS
up-solid down-solid
eye
Title
Date Published
Creator
Community Texts
Dec 30, 2019 Jan de Leeuw
texts
eye 5
favorite 0
comment 0
The paper recasts the minimization of the Gifi loss function as a pair of nonlinear eigenvalue problems of optimally scaled data (or a nonlinear singular value problem of optimally scaled data). The first and second derivatives of the eigenvalues with respect to the data transformations are computed.
Topics: Multivariate Analysis, R Programming
Community Texts
Nov 11, 2019 Jan de Leeuw
texts
eye 5
favorite 0
comment 0
Gifi's Eigenvalues
Topics: Multivariate Analysis, Gifi, Eigenvalues, R Programming
Community Texts
Oct 23, 2019 Jan de Leeuw
texts
eye 3
favorite 0
comment 0
These are notes to explain the update of the Gifi programs for nonlinear multivariate analysis, currently in progress.
Topics: Multivariate Analysis, Gifi, R Programming
Community Texts
Aug 27, 2019 Jan de Leeuw
texts
eye 10
favorite 0
comment 0
To study convergence of SMACOF we introduce a modification mSMACOF that rotates the configurations from each of the SMACOF iterations to principal components. This modification, called mSMACOF, has the same stress values as SMACOF in each iteration, but unlike SMACOF it produces a sequence of configurations that properly converges to a solution. We show that the modified algorithm can be implemented by iterating ordinary SMACOF to convergence, and then rotating the SMACOF solution to...
Topics: Multidimensional Scaling, SMACOF, Majorization, MM Algorithm, R Programming
Community Texts
Aug 21, 2019 Jan de Leeuw
texts
eye 11
favorite 0
comment 0
We review the continuity and differentiability properties of the stress loss function in MDS and the local and global convergence properties of the SMACOF algorithm.
Topics: Multidimensional Scaling, Stress, SMACOF, R Programming, Majorization, MM Algorithm
Community Texts
Jun 19, 2019 Jan de Leeuw
texts
eye 15
favorite 0
comment 0
If the n × p matrix X is a stationary point of the MDS loss function, then it is also the global minimum over the subspace of all n × p matrices with the same column space as X.
Topics: Multidimensional Scaling, Full-dimensional Scaling, SMACOF
Community Texts
Jun 9, 2019 Jan de Leeuw
texts
eye 5
favorite 0
comment 0
Majorization theory for families of majorizers is developed in the one-dimensional case. This generalizes the notion of a sharp quadratic majorizer discussed by De Leeuw and Lange (2009)
Topics: Optimization, Majorization, MM Algorithm
Community Texts
Jun 7, 2019 Jan de Leeuw
texts
eye 18
favorite 0
comment 0
The full-dimensional (metric, Euclidean, least squares) multidimensional scaling stress loss function is combined with a quadratic external penalty function term. The trajectory of minimizers of stress for increasing values of the penalty parameter is then used to find (tentative) global minima for low-dimensional multidimensional scaling. This is illustrated with several one-dimensional and two-dimensional examples.
Topics: Multidimensional Scaling, Global Optima, Full-dimensional Scaling, SMACOF, Exterior Penalty...
Community Texts
May 9, 2019 Jan de Leeuw
texts
eye 12
favorite 0
comment 0
This describes a C version of a infeasible primal-dual algorithm for positive definite quadratic programming with box constraints, proposed by Voglis and Lagaris. We also give a straightforward .C() interface for R.
Topics: Optimization, Quadratic Programming, R Programming
Community Texts
May 3, 2019 Jan de Leeuw
texts
eye 8
favorite 0
comment 0
We compute pseudo-confidence ellipses around MDS solutions, using a new fast implementation of the Hessian of the stress loss function.
Topics: Multidimensional Scaling, R Programming, Stability, Confidence Regions
Community Texts
Apr 29, 2019 Jan de Leeuw, Patrick Groenen, Patrick Mair
texts
eye 7
favorite 0
comment 0
In multidimensional scaling we map dissimilarity matrices into configurations for which stress is stationary. In inverse MDS we map configurations into dissimilarity matrices for which stress is stationary.
Topics: Multidimensional Scaling, SMACOF, Inverse Multidimensional Scaling, R Programming
Community Texts
Apr 29, 2019 Jan de Leeuw
texts
eye 13
favorite 0
comment 0
We discuss projection on the intersection of a polyhedral convex cone and a sphere, in particular when the target is in the polar cone. We also discuss projection on the double cone formed by the cone and its negative.
Topics: Optimization, Regression, Projection
Community Texts
Apr 24, 2019 Jan de Leeuw
texts
eye 11
favorite 0
comment 0
The stress loss function in metric multidimensional scaling is differentiable at local minima. In this note we generalize this result to more general functions of the distances.
Topics: Multidimensional Scaling, SMACOF
Community Texts
Apr 19, 2019 Jan de Leeuw
texts
eye 3
favorite 0
comment 0
We develop R and C code for Individual Difference Multidimensional Scaling, both for the INDSCAL and the IDIOSCAL case. In addition to the new SMACOF algorithms to minimize the stress loss function we use expressions for the second derivatives with respect to the group configuration and the individual weights to compute perturbation ellipsoids.
Topics: Multidimensional Scaling, INDSCAL, IDIOSCAL, Stability, Confidence Regions, R Programming, SMACOF
Community Texts
Apr 15, 2019 Jan de Leeuw
texts
eye 21
favorite 0
comment 0
In the usual forms of least squares nonlinear principal component analysis observed variables are quantified or transformed to optimize low-rank approximations. Thus NLPCA is linear PCA on optimally scaled variables. In this note we extend the approach by allowing for optimally scaled components.
Topics: Numerical Analysis, Principal Components, Alternating Least Squares, Block Relaxation,...
Community Texts
Apr 1, 2019 Jan de Leeuw
texts
eye 3
favorite 0
comment 0
We provide formulas for the convergence rate of majorization (and block relaxation) algorithms with constraints
Topics: Majorization, MM Algorithm, Convergence Rate
Community Texts
Mar 31, 2019 Jan de Leeuw
texts
eye 6
favorite 0
comment 0
We give an algorithm, with R code, to minimize the multidimensional scaling loss function proposed in Shepard’s 1962 papers. We show the loss function can be justified by using the classical rearrangement inequality, and we investigated its differentiability.
Topics: Multidimensional Scaling, Roger Shepard, Rearrangement Inequality, R Programming
Community Texts
Dec 26, 2018 Jan de Leeuw
texts
eye 11
favorite 0
comment 0
Using anarchic distances means using a different configuration for each dissimilarity. We give the anarchic version of the smacof majorization algorithm, and apply it to additive constants, individual differences, and scaling of asymmetric dissimilarities.
Topics: Multidimensional Scaling, Majorization, MM Algorithm, R Programming, SMACOF
Community Texts
Jun 19, 2018 Jan de Leeuw
texts
eye 9
favorite 0
comment 0
We present an R/C implementation of optimal simultaneous diagonalization of several real symmetric matrices using Jacobi plane rotations, with compact triangular storage of symmetric matrices.
Topics: Numerical Analysis, Arrays, Multiway Analysis, Simultaneous Diagonalization, Jacobi Rotations, R...
Community Texts
May 24, 2018 Jan de leeuw
texts
eye 13
favorite 0
comment 0
Earlier papers have shown that stress is differentiable at local minima if certain conditions on the weights and dissimilarities are satisfied. In this note we show the result remains true without these additional conditions.
Topics: Multidimensional Scaling, Stress, Differentiability
Community Texts
May 7, 2018 Jan de Leeuw
texts
eye 4
favorite 0
comment 0
The positive orthant method tries to find solutions to consistent systems of inequalities, and approximate solutions to inconsistent systems, by maximizing a fit measure based on the sign function and the absolute value function. We concentrate on systems of linear inequalities and develop a convergent majorization algorithm.
Topics: Optimization, Inequalities, Majorization, MM Algorithm, R Programming
Community Texts
May 4, 2018 Jan de Leeuw
texts
eye 23
favorite 0
comment 0
We discuss two different even and convex non-negative smooth approximations of the absolute value function and apply them to construct MM algorithms for least absolute deviation regression. Both uniform and sharp quadratic majorizations are constructed. As an example we use the Boston housing data. In our example sharp quadratic majorization is typically 10-20 times as fast as uniform quadratic majorization.
Topics: Optimization, MM Algorithm, R Programming, Majorization
Community Texts
Apr 9, 2018 Jan de Leeuw
texts
eye 14
favorite 0
comment 0
The Cauchy-Schwartz majorization of the distance function in SMACOF is replaced by a majorization of the squared distance function. This leads to an interesting SMACOF alternative, which we call SMOCAF.
Topics: Multidimensional Scaling, Majorization, MM Algorithm, R Programming, SMACOF
Community Texts
Feb 24, 2018 Jan de Leeuw, C. Roger Nance
texts
eye 10
favorite 0
comment 0
This paper is a non-technical and mostly graphical introduction to Homogeneity Analysis, also known as Multiple Correspondence Analysis. It is meant as an explanation and justification of a non-standard application of Correspondence Analysis to an example from archeology.
Topics: Multivariate Analysis, Homogeneity Analysis, Gifi, Multiple Correspondence Analysis, Archeology
Community Texts
Oct 10, 2017 Jan de Leeuw
texts
eye 12
favorite 0
comment 0
The smacof algorithm for (metric, Euclidean, least squares) multidimensional scaling is rewritten so that all computation is done in C, with only the data management, memory allocation, iteration counting, and I/O handled by R. All symmetric matrices use compact, lower triangular, column-wise storage. Second derivatives of the loss function are provided, but non-metric scaling, individual differences, and constraints still have to be added.
Topics: Multidimensional Scaling, SMACOF, Majorization, MM Algorithm, R Programming
Community Texts
Aug 31, 2017 Jan de Leeuw
texts
eye 9
favorite 0
comment 0
We give R, C, and R->C code to access lineary stored multidimensional arrays and compactly stored multidimensional super-symmetric arrays.
Topics: Numerical Analysis, Multidimensional Arrays, R Programming
Community Texts
Aug 6, 2017 Jan de Leeuw
texts
eye 2
favorite 0
comment 0
We define fDistances, which generalize Euclidean distances, squared distances, and log distances. The least squares loss function to fit fDistances to dissimilarity data is fStress. We give formulas and R/C code to compute partial derivatives of orders one to four of fStress, relying heavily on the use of Faà di Bruno’s chain rule formula for higher derivatives.
Topics: Multidimensional Scaling, SMACOF, R Programming
Community Texts
texts
eye 8
favorite 0
comment 0
The Jacobi method for computing eigenvalues and eigenvectors of a symmetric matrix is implemented in C using column-wise compact storage of the lower triangle. The complied C code can be loaded into R using the .C() interface. We compare the C implementation with an earlier version in pure R, and with the built-in eigen function in R.
Topics: Numerical Analysis, Linear Algebra, Jacobi Method, Eigenvalues, R Programming
Community Texts
Jun 3, 2017 Jan de Leeuw
texts
eye 16
favorite 0
comment 0
We give a majorization algorithm for weighted least squares low-rank matrix approximation, a.k.a. principal component analysis. The loss function has one non-negative weight for each squared residual. A quadratic programming method is used to compute optimal rank-one weights for the majorization scheme.
Topics: Numerical Analysis, Linear Algebra, Majorization, MM Algorithm, R Programming
Community Texts
May 26, 2017 Jan de Leeuw
texts
eye 19
favorite 0
comment 0
In many situations in numerical analysis least squares loss functions with diagonal weight matrices are much easier to minimize than least square loss functions with full positive semi-definite weight matrices. We use majorization to replace problems with a full weight matrix by a sequence of diagonal weight matrix problems. Diagonal weights which optimally approximate the full weights are computed using a simple semi-definite programming procedure.
Topics: Optimization, Weighted Least Squares, Majorization, MM Algorithm, R Programming
Community Texts
May 21, 2017 Jan de Leeuw
texts
eye 3
favorite 0
comment 0
We give necessary and sufficient conditions for solvability of A j = XW j X 0 , with the A j are m given positive semi-definite matrices of order n. The solution X is n × p and the m solutions W j are required to be diagonal, positive semi-definite, and adding up to the identity. We do not require that p ≤ n.
Topics: Numerical Analysis, Arrays, Multiway Analysis, Simultaneous Diagonalization
Community Texts
May 5, 2017 Jan de Leeuw
texts
eye 6
favorite 0
comment 0
A general form of linear factor analysis is defined, and presented as a method to factor a data matrix, similar in many respects to principal component analysis. We discuss necessary and sufficient conditions for solvability of the factor analysis equations and give a constructive method to compute all solutions. A follow up paper will present the corresponding algorithm.
Topics: Multivariate Analysis, Factor Analysis
Community Texts
Apr 17, 2017 Jan de Leeuw
texts
eye 17
favorite 0
comment 0
A brief introduction to spline functions and B-splines, and specifically to monotone spline functions – with code in R and C and with some applications.
Topics: Numerical Analysis, B Splines, Monotone Splines, R Programming
Community Texts
Apr 1, 2017 Jan de Leeuw
texts
eye 3
favorite 0
comment 0
A C implementation of Kruskal’s up-and-down-blocks monotone regression algorithm for use with .C() is extended to include the three classic ways of handling ties. It is then compared with other implementations.
Topics: Optimization, Isotone Regression, Monotone Regression, R Programming
Community Texts
Mar 30, 2017 Jan de Leeuw
texts
eye 9
favorite 0
comment 0
A C implementation of Kruskal’s up-and-down-blocks monotone regression algorithm for use with .C(), and a comparison with other implementations.
Topics: Optimization, Isotone Regression, Monotone Regression, R Programming
Community Texts
Mar 22, 2017 Jan de Leeuw
texts
eye 6
favorite 0
comment 0
We use the system qsort to write a routine that produces both the sort an the order of a vector of doubles.
Topics: Numerical Analysis, Sorting, R Programming
Community Texts
Jan 16, 2017 Jan de Leeuw
texts
eye 4
favorite 0
comment 0
We give an algorithm, with R code, to minimize the multidimensional scaling stress loss function under the condition that some or all of the fitted distances are between given positive upper and lower bounds. This paper combines theory, algorithms, code, and results of De Leeuw (2017b) and De Leeuw (2017a).
Topics: Multidimensional Scaling, R Programming, SMACOF, MM Algorithm, Majorization
Community Texts
Jan 13, 2017 Jan de Leeuw
texts
eye 4
favorite 0
comment 0
We give an algorithm, with R code, to minimize the multidimensional scaling stress loss function under the condition that some or all of the fitted distances are smaller than given upper bounds.
Topics: Multidimensional Scaling, R Programming, Majorization, MM Algorithm, SMACOF
Community Texts
Jan 13, 2017 Jan de Leeuw
texts
eye 10
favorite 0
comment 0
We give an algorithm, with R code, to minimize the multidimensional scaling stress loss function under the condition that some or all of the fitted distances are larger than given positive lower bounds. This paper is a companion to De Leeuw (2017). We also give some interesting majorization theory.
Topics: Multidimensional Scaling, R Programming, Majorization, MM Algorithm, SMACOF
Community Texts
Jan 3, 2017 Jan de Leeuw
texts
eye 11
favorite 0
comment 0
We give a quick and dirty, but reasonably safe, algorithm for the minimization of a convex quadratic function under convex quadratic constraints. The algorithm minimizes the Lagrangian dual by using a safeguarded Newton method with non-negativity constraints.
Topics: Optimization, Quadratic Programming, Quadratic Constraints, R Programming
Community Texts
Dec 20, 2016 Jan de Leeuw
texts
eye 4
favorite 0
comment 0
We discuss the problem of finding an approximate solution to an overdetermined system of linear inequalities, or an exact solution if the system is consistent. Theory and R code is provided for fouractive set methods for non-negatively constrained least squares, one uses alternating least squares, and one uses a nonsmooth Newton method.
Topics: Optimization, Linear Inequalities, R Programming, Alternating Least Squares
Community Texts
Dec 14, 2016 Jan de Leeuw
texts
eye 2
favorite 0
comment 0
We construct piecewise quadratic majorizers for minimax problems. This is appled to finding roots of cubics. An application to a Chebyshev versions of MDS loss is also outlined.
Topics: Optimization, Minimax, Majorization, MM Algorithm, R Programming
Community Texts
Dec 11, 2016 Jan de Leeuw
texts
eye 7
favorite 0
comment 0
We illustrate uniform quadratic majorization, sharp quadratic majorization, and sublevel quadratic majorization using the example of a univariate cubic.
Topics: Optimization, Majorization, MM Algorithm, R Programming
Community Texts
Dec 2, 2016 Jan de Leeuw
texts
eye 6
favorite 0
comment 0
We study the convergence rate of the ELEGANT algorithm for squared distance scaling by using both observed convergence rates and an analytical expression for the derivative of the algorithmic map.
Topics: Multidimensional Scaling, Squared Distance Scaling, Convergence Rate, R Programming
Community Texts
Nov 23, 2016 Jan de Leeuw
texts
eye 10
favorite 0
comment 0
In De Leeuw (2008) we studied the derivatives of the least squares rank p approximation in the case of general rectangular matrices. We modify these results for the symmetric positive semi-definite case, using basically the same derivation. We apply the formulas to compute an expression for the convergence rate of Thomson’s iterative principal component algorithm for factor analysis.
Topics: Numerical Analysis, Low Rank Approximation, Convergence Rate, R Programming
Community Texts
Nov 10, 2016 Jan de Leeuw, Patrick Groenen, Raoul Pietersz
texts
eye 15
favorite 0
comment 0
We reproduce the 1975 derivation of the alternating least squares algorithm for squared distance scaling, from an internal report that got lost in the folds of time. In addition, we present a derivation and a substantial speed improvement based on majorization.
Topics: Multidimensional Scaling, Squared Distance Scaling, Majorization, MM Algorithm, R Programming,...
Community Texts
Nov 4, 2016 Jan de Leeuw
texts
eye 6
favorite 0
comment 0
This short note shows that all block relaxation algorithms can be formulated as majorization algorithms. The result is mostly a curiosity, without any obvious practical applications
Topics: Majorization, Block Relaxation, Optimization, MM Algorithm
Community Texts
Jun 29, 2016 Jan de Leeuw
texts
eye 12
favorite 0
comment 0
A low-dimensional multidimensional scaling example is used to illustrate properties of the stress loss function and of different iteration methods.
Topics: Multidimensional Scaling, Stress, SMACOF, Majorization, MM Algorithm, Newton Method, R Programming,...
Community Texts
Jun 8, 2016 Jan de Leeuw
texts
eye 8
favorite 0
comment 0
In Multidimensional Scaling we sometimes find that stress does not decrease if we increase dimensionality. This is explained in this note by using the Gower rank. Some examples with small Gower rank are analyzed.
Topics: Multidimensional Scaling, SMACOF, Full-dimensional Scaling
Community Texts
Mar 23, 2016 Jan de Leeuw
texts
eye 6
favorite 0
comment 0
This is R/C code for a sequence of principal pivot transforms of a matrix, with applications to least squares, inversion, and determinants.
Topics: Numerical Analysis, Linear Algebra, Principal Pivots, R Programming
Community Texts
Mar 21, 2016 Jan de Leeuw
texts
eye 15
favorite 0
comment 0
We present R/C code for rank-revealing QR decomposition with application to generalized inverse, linear equality systems, least squares solutions, and null spaces.
Topics: Numerical Analysis, Linear Algebra, R Programming
Community Texts
Mar 14, 2016 Jan de Leeuw, Patrick Groenen, Patrick Mair
texts
eye 11
favorite 0
comment 0
rStress is the weighted sum-of-squares of the differences between dissimilarities and r-th powers of squared distances. We derive a majorization algorithm for the multidimensional scaling loss function rStress, with r small.
Topics: Multidimensional Scaling, rStress, Majorization, MM Algorithm, R Programming
Community Texts
Mar 7, 2016 Jan de Leeuw, Masanao Yajima
texts
eye 29
favorite 0
comment 0
R versions of the array manipulation functions of APL are presented. We do not translate the system functions or other parts of the runtime. Also, the current version has does not have the nested arrays of APL-2.
Topics: APL, R Programming, C Programming
Community Texts
Feb 23, 2016 Jan de Leeuw
texts
eye 13
favorite 0
comment 0
In this note we give simple functions, using the .C interface in R, to compute the next permutation or combination in the lexicographic order. 
Topics: Numerical Analysis, Permutations, Combinations, R Programming
Community Texts
Feb 23, 2016 Jan de Leeuw
texts
eye 14
favorite 0
comment 0
Implementation in R/C of Gram-Schmidt orthogonalization.
Topics: Numerical Analysis, Linear Algebra, R Programming
Community Texts
Feb 15, 2016 Jan de Leeuw, Patrick Groenen, Patrick Mair
texts
eye 3
favorite 0
comment 0
We study the properties of stress at points where some distances are zero and the configuration matrix is singular.
Topics: Multidimensional Scaling, Stress, Local Minima
Community Texts
Feb 8, 2016 Jan de Leeuw, Patrick Groenen, Patrick Mair
texts
eye 12
favorite 0
comment 0
rStress is the weighted sum-of-squares of the differences between dissimilarities and r-th powers of squared distances. We investigate if De Leeuw (1984) on the differentiability of stress (i.e. rstress with r = ½) at a local minimum generalizes to other values of r.
Topics: Multidimensional Scaling, rStress, Differentiability
Community Texts
Jan 25, 2016 Jan de Leeuw, Patrick Groenen, Patrick Mair
texts
eye 11
favorite 0
comment 0
This paper discusses full-dimensional scaling, which is multidimensional scaling of n points in n dimensions.
Topics: Multidimensional Scaling, SMACOF, R Programming, Full-dimensional Scaling
Community Texts
Jan 20, 2016 Jan de Leeuw, Patrick Groenen, Patrick Mair
texts
eye 12
favorite 0
comment 0
rStress is the weighted sum-of-squares of the differences between dissimilarities and r-th powers of squared distances. This note gives formulas, code, and applications for the second derivatives of rStress.
Topics: Multidimensional Scaling, rStress, Derivatives, R Programming
Community Texts
Jan 19, 2016 Jan de Leeuw
texts
eye 13
favorite 0
comment 0
This is an R wrapper for Cran's Fortran function AMALGM from Applied Statistics. 1980. At the R level it adds the three ways of dealing with ties.
Topics: Optimization, Quadratic Programming, Isotone Regression, Monotone Regression, R Programming
Community Texts
Jan 14, 2016 Jan de Leeuw, Patrick Groenen, Patrick Mair
texts
eye 7
favorite 0
comment 0
rStress is the weighted sum-of-squares of differences between dissimilarities and r-th power of squared distances. We give a majorization algorithm, and R code, for both r > ½ and r <= ½.
Topics: Multidimensional Scaling, rStress, Majorization, MM Algorithm, R Programming
Community Texts
Dec 16, 2015 Jan de Leeuw
texts
eye 17
favorite 0
comment 0
An analysis of the "horseshoe effect" in Multiple Correspondence Analysis. Where do they come from, and are they harmful ?
Topics: Multivariate Analysis, Gifi, R Programming, Correspondence Analysis
Community Texts
Dec 1, 2015 Jan de Leeuw
texts
eye 18
favorite 0
comment 0
The Burt matrix collects all bivariate cross tables, and/or covariance matrices, of m variables in a single matrix. Various forms of canonical analysis based on the Burt matrix are discussed.
Topics: Multivariate Analysis, R Programming, Gifi, Correspondence Analysis
Community Texts
Nov 12, 2015 Jan de Leeuw
texts
eye 2
favorite 0
comment 0
The least squares loss function used in Gifi is rewritten in a slightly different form, with considerable algorithmic implications. Code and examples in R are included.
Topics: Multivariate Analysis, Gifi, R Programming
Community Texts
May 25, 2015 Jan de Leeuw
texts
eye 14
favorite 0
comment 0
Suppose we have a dataframe with n observations on m variables, i.e. m vectors in ℝ^n. We want to transform the m variables in such a way that a particular aspect of the correlation matrix of the variables is maximized. An aspect is a real valued function f defined on the space of correlation matrices.
Topics: Multivariate Analysis, Aspects, Gifi, R Programming